arXiv: 1501.01144v2 [q-bio.NC] 2 Jan 2016 


Airborne Ultrasonic Tactile Display BCI 

Katsuhiko Hamada 1 * Hiromu Mori 2 ’! Hiroyuki Shinoda 1 
and Tornasz M. Rutkowski 2,3, ki 


1 The University of Tokyo, Tokyo, Japan 
2 Life Science Center of TARA, University of Tsukuba, Tsukuba, Japan 
3 RIKEN Brain Science Institute, Wako-shi, Japan 
tomek@bci-lab.info & http://bci-lab.info/ 


Abstract 

This chapter presents results of our project, which studied whether contactless and 
airborne ultrasonic tactile display (AUTD) stimuli delivered to a user’s palms could serve 
as a platform for a brain computer interface (BCI) paradigm. We used six palm positions 
to evoke combined somatosensory brain responses to implement a novel contactless tactile 
BCI. This achievement was awarded the top prize in the Annual BCI Research Award 2014 
competition. This chapter also presents a comparison with a classical attached vibrotac- 
tile transducer-based BCI paradigm. Experiment results from subjects performing online 
experiments validate the novel BCI paradigm. 


1 Introduction 

State-of-the-art brain computer interfaces (BCIs) are usually based on mental, visual or au¬ 
ditory paradigms, as well as body movement imagery paradigms, which require extensive user 
training and good eyesight or hearing. In recent years, alternative solutions have been proposed 
to make use of the tactile modality [1, 2, 3] to enhance BCI efficiency. The concept reported 
in this chapter further extends the brain’s somatosensory channel by applying a contactless 
stimulus generated with an airborne ultrasonic tactile display (AUTD) [4]. This is an expanded 
version of a conference paper published by the authors [5]. 

The rationale behind the use of the AUTD is that, due to its contactless nature, it allows 
for a more hygienic application, avoiding the occurrence of skin ulcers (bedsores) in patients 
in a locked-in state (LIS). The AUTD permits a less complex application of the BCI for the 
caregivers comparing to classical attached vibrotactile transducers’ setups. 

This chapter reports very encouraging results with AUTD-based BCI (autdBCI) compared 
to the classical paradigm using vibrotactile transducer-based oddball (P300 response-based) 
somatosensory stimulus (vtBCI) attached to the user’s palms [3]. 

The rest of the chapter is organized as follows. The next section introduces the methods 
used in the study. The results obtained in online experiments with 13 healthy BCI users are 
then discussed. Finally, conclusions are drawn and directions for future research are outlined. 


2 Methods 

Thirteen male volunteer BCI users participated in the reported in this chapter experiments. 
The users’ mean age was 28.54, with a standard deviation of 7.96 years. The experiments were 
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performed at the Life Science Center of TARA, University of Tsukuba, at the University of 
Tokyo and at RIKEN Brain Science Institute, Japan. The online (real-time) EEG autdBCI 
and vtBCI paradigm experiments were conducted in accordance with the WMA Declaration of 
Helsinki - Ethical Principles for Medical Research Involving Human Subjects and the procedures 
were approved and designed in agreement with the ethical committee guidelines of the Faculty 
of Engineering, Information and Systems at University of Tsukuba, Japan (experimental per¬ 
mission 2013.R7). 

The AUTD stimulus generator produced vibrotactile contactless stimulation of the human 
skin via the air using focused ultrasound [4, 6]. The effect was achieved by generating an 
ultrasonic radiation static force produced by intense sound pressure amplitude (a nonlinear 
acoustic phenomenon). The radiation pressure deformed the surface of the skin on the palms, 
creating a virtual touch sensation. An array of ultrasonic transducers mounted on the AUTD 
(see Figure 1) created the focused radiation pressure at an arbitrary focal point by choosing a 
phase shift of each transducer appropriately (the so-called phased array technique). Modulated 
radiation pressure created a sensation of tactile vibration similar to the one delivered by classical 
vibrotactile transducers attached to the user’s palms, as shown in Figure 2. The AUTD device 
developed by the authors [1,6] (see Figure 1) adhered to ultrasonic medical standards and did 
not exceed the permitted skin absorption levels (approximately 40 times below the permitted 
limits). The effective vibrotactile sensation was set to 50 Hz [6] to match with tactile skin 
mechanoreceptors’ frequency characteristics and the notch filter that EEG amplifiers use for 
power line interference rejection. 

As a reference, in the second vtBCI experiment, contact vibrotactile stimuli were also applied 
to locations on the users’ palms via the transducers HIHX09C005-8. Each transducer in the 
experiments was set to emit a square acoustic frequency wave at 50 Hz, which was delivered 
from the ARDUINO micro-controller board with a custom battery-driven and isolated power 
amplifier and software developed in-house and managed from a MAX 6 visual programming 
environment. 

The two experiment set-ups above are presented in Figures 2 and 3. Two types of experi¬ 
ments were performed with the volunteer healthy users. Experiments with the target paralyzed 
users are planned as a followup of the current pilot project. Psychophysical experiments with 
foot-button-press responses were conducted to test uniform stimulus difficulty levels from re¬ 
sponse accuracy and time measurements. The subsequent tactile oddball online BCI EEG 
experiments evaluated the autdBCI paradigm efficiency and allowed for a comparison with 
the classical skin contact-based vtBCI reference. In both the above experiment protocols, the 
users were instructed to spell sequences of six digits representing the stimulated positions on 
their palms. The training instructions were presented visually by means of the BCI2000 [ \ 
and MAX 6 programs with the numbers 1 — 6 representing the palm locations as depicted in 
Figure 2. 

The EEG signals were recorded with the g.USBamp amplifier system from g.tec Medical 
Engineering GmbH, Austria, using 16 active g.LADYbird electrodes. The electrodes were at¬ 
tached to the head locations: Cz, Pz, P3, Pj, C3, Cj, CP5, CP6, PI, P2, POz, Cl, C2, 
FC1, FC2, and FCz, as in the 10/10 extended international system. The ground electrode was 
attached to the FPz position, and the reference was attached to the left earlobe. No electro¬ 
magnetic interference was observed from the AUTD or vibrotactile transducers operating with 
frequencies notch filtered together with power line interference from the EEG. The EEG sig¬ 
nals captured were processed online with an in-house extended BCI2000-based application [7], 
using a stepwise linear discriminant analysis (SWLDA) classifier [8] with features drawn from 
0 — 800 ms ERP intervals decimated by a factor of 20. 



Figure 1: The AUTD array with ultrasonic transducers used to create the contactless tactile 
pressure sensation. 



Figure 2: User’s palms with attached vibrotactile transducers used in vtBCI experiments. Each 
stimulus location reflects a different digit. 




Figure 3: A user during the autdBCI experiment with both palms placed under the AUTD 
array with ultrasonic transducers. 


The stimulus length and inter-stimulus-interval were set to 400 ms, and the number of 
epochs to average was set to 15. The EEG recording sampling rate was set at 512 Hz, and 
the high and low pass filters were set at 0.1 Hz and 60 Hz, respectively. The notch filter to 
remove power line interference removed activity between 48 ~ 52 Hz. Each user performed 
three experiment runs (randomized 90 targets and 450 non-targets each). As feedback, the 
spelled numbers (palm position assigned digits as in Figure 2) were shown on a display to the 
user. 


3 Results 

The grand mean averaged evoked responses to targets and non-targets are depicted together 
with standard error bars in Figure 4 and as matrices with an area under the curve (AUC) 
analysis for feature separability in Figure 5. The BCI six digit sequences spelling accuracy 
analyses for both experiments for the various averaging options are summarized in Figure 6. The 
chance level was 16.6%. The mean six digit sequence spelling accuracies for 15-trial averaged 
ERPs were 63.8% and 69.4% for autdBCI and vtBCI, respectively. The maximum accuracies 
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Figure 4: The autdBCI (blue - targets; red - non-targets) and vtBCI (green - targets; black 
non-targets) grand mean averaged ERP responses, together with standard error bars. 


were 78.3% and 84.6% respectively. The differences were not significant, supporting the concept 
of using autdBCIs. Flowever, a single trial classification offline analysis of the collected responses 
resulted with the best obtained accuracies of 83.0% for autdBCI and 53.8% for vtBCI, leading 
to a possible 19.2 bit/min and 7.9 bit/min, respectively. 

In the case of the autdBCI, only a single user’s results were bordering on the level of 
chance, and four subjects attained 100% (10 trials averaging). On average, lower accuracies 
were obtained with the classical vtBCI, with which three users bordered on the level of chance, 
and only one user scored 100% accuracy level in SWLDA-classified averaged responses. 
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Figure 5: The autdBCI grand mean averaged ERP responses, shown as matrix plots for targets 
in the top panel; non-targets in the middle; and area under the curve (AUC) of the response 
discriminability analysis (AUC > 0.5 marks the discriminable latencies). 


4 Conclusions 

This study demonstrates results obtained with a novel six-command-based autdBCI paradigm. 
We compared the results with a classical vibrotactile transducer stimulus-based paradigm. The 
experiment results obtained in this study confirm the validity of the contactless autdBCI for 
interactive applications and the possibility to further improve the results through single trial- 
based SWLDA classification. 

The EEG experiment with our paradigm confirms that contactless (airborne) tactile stim¬ 
uli can be used to create six command- based BCIs in real time. A short demo with online 
application of the paradigm to robotic arm control is available on YouTube [ ]. 

The results presented offer a step forward in developing and validating novel neurotechnology 
applications. Since most users did not achieve very high accuracy during online BCI operation, 
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Figure 6: Averaged autdBCI and vtBCI spelling accuracy across six digits, colour coded for 
each user with standard error bars. 


especially with only a few trials, the current paradigm obviously requires improvement and 
modification. These requirements determine the major lines of study for future research. 

However, even in its current form, the proposed autdBCI can be regarded as a practical so¬ 
lution for LIS patients (locked into their own bodies despite often intact cognitive functioning), 
who cannot use vision or auditory-based interfaces due to sensory or other disabilities. The 
reported autdBCI project was awarded The BCI Annual Research Award 2014 for “A fasci¬ 
nating new idea never explored before,” according to the Chairman of the Jury for the Annual 
BCI Research Award 2014, Prof. Gernot R. Mueller-Putz from the Institute for Knowledge 
Discovery, Graz University of Technology, Austria. 
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